Music and Movies | Computational Musicology 2021
Esther L. Bakels 
Almost every movie has music in it. These soundtracks often play in the background of a movie, so you would expect the music is calmer than music you’d listen to. On the other hand, maybe there are some features in the music that are not so different from your own music. I made three playlists:
My goal is to examine different features, compare them to one another and search for a feature that isn’t that different from my music. When I find that feature, I hope to find the theme song that corresponds the most to my favorite songs.I would also like to look into the Theme songs and More soundtracks playlists together and investigate the songs in it.
To investigate which feature of the spotify API can classify the songs from my corpus, a k-NN model is trained. At the left we can see a mosaic of the trained model that shows how well it performed. Next to the mosaic a graph is shown with the different features that were the most effective for the classification.
The mosaic of the k-NN shows a clear difference between film music and my favorite songs.The left upper block is bigger than the left bottom block and the right bottom block us (way) bigger than the right upper block. This means that the prediction are pretty accurate. The precise numbers of the mosaic are shown in the table below:
To know how well the model did, precision and recall can be calculated. The recall, which is the true positives among all real positives, is pretty high. The precision, the true positives among all predicted positives, is also pretty high. This means that the model did very well.
We can see in the graph that the feature loudness, instrumentalness and energy were the most significant features, together with c01 (timbre component 1). A feature that was not really effective was tempo.
The features energy and tempo will be used in further visualisations.
This cluster representation shows loudness versus timbre component 1. Both clusters are directly proportional, which means that of loudness goes up, c01 also goes up. The playlist of my favorite song is clustered at the right top, while the soundtracks are more scattered over the diagonal. There is a soundtrack “Misirlou” is on top, this track also has a lot of energy.
The median of the different categories is somewhat the same. The 50% around the median is differs more in category C, my songs. Category B has the lowest median, but an outlier with the highest tempo of all categories. The outlier is The Portrait - James Horner.
The median of the different catogories is somewhat the same for theme songs and more soundtracks, but way higher for my songs. The median of category A is higher than B, this is maybe because theme songs are the main song of the movie. Category B also includes songs that are used as more background music in movies. The outlier “The Portrait” from boxplot 1 is this time one of the lowest few.
The median of tempo is plotted over the years. In boxplot 1, category A, we saw that the 50% around the median was small. In this graph, we can see that the 50% is so small because the tempo did not really change over the years.
The median is plotted over the years. In boxplot 2, category A, we saw that the 50% around the median was a lot bigger than boxplot 1. In this graph, we can see that the 50% is so big because the tempo did really change over the years. In the early stages, energy was at its highest point, but decreased rapidly. but around the 2000, energy has a turning point, and started to increase again.
If you click on A and B, only C will show. You’ll see that a red cluster is formed in the right side of the plot. A and B together are more active on the left side.You don’t see clusters forming in the up or down side. Tempo seems equally distributed for all the three playlists, unlike the other feature energy. We also saw this in the visualisation per feature.
We now have an overall view of the playlists. The features Tempo and Energy showed different relations between the playlists. In the feature Tempo, the medians of all playlists were close together. Energy on the other hand, showed that the median of My Songs was higher than that of the soundtrack playlists. To understand the corpus better, we are going to dive in the soundtrack playlists (Theme songs and More Soundtracks) and hopefully discover some interesting findings. Songs that do not fit in the bunch are interesting to investigate. I would also like to investigate the similarity between playlists, so I will also investigate a track that blends in.
The boxplots of Tempo showed that The Portrait - James Horner was an outlier in the More Soundtracks playlist.It wasn’t only an outlier in that playlist, it deviated from all three playlists. A boxplot also shows a median, a track can be seen as the midpoint of the dataset. But to find a song that blends in, I will use another method that I like more in the column on the right.
The Portrait is a song from the More Soundtracks playlist, so isn’t a theme song. This song plays is the movie Titanic. In further investigations I will use this track as an outlier of my corpus.
As was shown in the visualisations, the (median) tempo of the different playlists didn’t really differ from each other. To know what soundtrack would fit the best in my favorite songs playlist, the average tempo of my favorite songs were calculated.
The mean tempo of my favorite songs is: 119.9941.
One of the songs in the theme song playlist should correspond the most with this mean tempo.
The corresponding track by this song is:
| track.name |
|---|
| Alice’s Theme |
The track Alice’s Theme is from the composer Danny Elfman, and (as you can see in the title) plays in the movie Alice in Wonderland.
On the left are the chromagrams from Alice’s Theme and The Portrait shown.
Alice’s Theme - Danny Elfman: The yellow places are the pitches that occur the most in the tracks. Alice’s Theme has a lot of “C” in the first few seconds. Soon “A” takes over. The chromagram overall looks a bit messy because all the different pitches are used.
The Portrait - James Horner: The pitches used in this track are more clear. In the first three minutes of the song, seven different pitches are used (“C”, “D”, “E”, “F”, “G”, “A” and “A#”). After three minutes, “D”, “E” and “A” stay present, but the other four pitches go one up.
On the left we see the very different cepstrograms from Alice’s Theme and The Portrait that are used to investigate timbre in a musical piece. Timbre, that is different from pitch as we saw in the chromagrams, looks at the sound colour of the note. Timbre can seperate different musical instruments that play in the same pitch for example.
The graphs show different timbre components over time. According to the spotify API documentation, the first timbre component represents the average loudness of the segment; second emphasizes brightness; third is more closely correlated to the flatness of a sound; fourth to sounds with a stronger attack; etc.
Alice’s Theme - Danny Elfman: A lot of the timbre components are used in this piece. The second timbre component looks the most present, but also the first, third and fifth are almost just as present.
The Portrait - James Horner: Just like Alice’s Theme, the second timbre component is most present. But the other timbre components are not present at all.
That the cepstrograms look so different ca be explained. Alice’s Theme has a lot more variation, it contains voices and different instruments. The Portrait on the other hand, is a piece with only piano.
On the left are the self-similarity matrices from Alice’s Theme and The Portrait shown.
Alice’s Theme You can see a chessboard pattern, which means there is homogeneity in this soundtrack. Homogeneity means that passages or sections in the piece are persistent and contain similar features (tempo, instrumentation etc.) with some other musical property in the song. Next to homogeneity based approaches, you also have repetition based approaches, that shows the structure of a piece, and especially focuses on recurring patterns/sections in the song. This would show some diagonal lines in the graph, which we don’t see in Alice’s Theme.
The Portrait In this piece, we see some big blue blocks, the one darker blue than the other. The sections in the piece are very similar the one another. Only the yellow lines in the SSM shows that the end is very different from the rest of the song. This is because this song has a fade out at the end, were the last seconds are completely silent. A long silence isn’t in other parts of the song of course, so a yellow line appears.
On the left the we see the chordogram and keygram of the track Also sprach Zarathustra (sonnenaufgang) - Richard Strauss is shown.
Also sprach Zarathustra is a 9 piece symphony composed by Richard Strauss.The first part was used in the movie 2001: A space Odyssey. In this symphony, the C-chord stands for nature and the B-chord for humanity. If we look at the chordogram, you don’t really see those chords. The Chordogram shows an outro in D major. The keygram shows that the outro is in D minor key.
I was interested in the chromagram, so that is shown too. Here you can see C present. A lot of nature vibes in the sunrise according to the chromagram.
On the left the we see the chordogram and keygram of the track Hedwig’s Theme - John Williams is shown.
This track is for me the most nostalgic and iconic theme song there is. This theme is the leitmotif that occurs in all of the eight Harry Potter films and the spin-off Fantastic Beasts, although not usually in its unaltered state.
The keygram shows that this track starts in D flat major and changes to D major after two minutes. The chordogram also shows an intro in D flat major, but after two minutes the chords C minor, F major and D7 show up.
It could be interesting, in further investigation, to compare track with the variation in Fantastic Beasts. This portfolio will not show that, because that song wasn’t in the corpus.
On the left the we see the chordogram and keygram of the track The Good, The Bad and The Ugly - Ennio Morricone is shown.
This song is the main theme from one of the most popular western movies. The main theme contains two types of flutes and human voices, that together should resemble the howling of a coyote.
The chords most apparent in the chordogram are E flat 7 and F7. After one minute D major in the chordogram, and also in the keygram.
It could be interesting, in further investigation, to compare track with the other songs from western movies. This portfolio will show that.
On the left the we see the tempogram of the track Alice’s Theme - Danny Elfman is shown. A tempogram indicates for each time instance the local relevance of a specific tempo for a given music recording.
You can see three yellow lines at 100, 250 and 350 BPM. The real tempo is 100 BPM. The other two lines, 250 and 350 BPM, are really fast. The lines are consistent horizontal lines, which means that the tempo doesn’t really variate a lot.
Tempograms
On the left the we see the tempograms of the tracks The Good, The Bad and The Ugly - Ennio Morricone and For a Few Dollars More - Ennio Morricone are shown.
Both soundtracks are made by Ennio Morricone and are from the Dollars Trilogy, an Italian film series consisting of three Spaghetti Western films.
The first graph has some yellow lines. You can see a line at 200 BPM that increases slightly and you also see what activity just above 400. The second soundtrack starts after about 1 minute, and you can see it has around the same BPM as the first one. The tempo in these grams aren’t as consistent as Alice’s Theme.
Novelty
On the left the we also see the novelty of the track For a Few Dollars More - Ennio Morricone around the beginning of the song.The song starts at 1:06. You can see that the yellow vertical line in the tempogram represents the silence in the novelty graph.
We saw in the machine learning model some components in music could differentiate the different playlists very well. The features loudness, instrumentalness and energy were very effect classifying the songs. Other features, for example tempo, did a lesser job.
When we looked further into these features, boxplots showed us the same thing as the model. The average tempo of the playlists wasn’t different at all. But when we looked at energy, the boxplots showed a significant difference.
A song that is similar to all playlist was Alice’s Theme, which tempo corresponded to the mean tempo of my favorite songs. A song that was very different from all other playlists was The Portrait, that was an outlier when looking at tempo.
Part to of this portfolio zoomed in on different songs of the movie playlists. We saw that every song was unique in its own way. But tempo was again a feature were similarities could be seen. When looking at the Wild West soundtracks of Ennio Moricone, the tempo was approximately the same in both tracks.